<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN">
<html>
  <head>
    <title>Installation, Deployment and Run</title>
  </head>

  <body>
    <h1>Installation, Deployment and Run</h1>

<b><font color="#0000ff">Overview:</font></b><br>
Sun Grid Engine (SGE) can be used as a globus jobmanager.  The
jobmanager type for SGE is `grd'.  The globus toolkit includes job
script files for SGE (globus-script-grd-{submit,poll,rm,queue*}).
<br><br>


<b><font color="#0000ff">Installation, deployment and run:</font></b><br>
In order to enable SGE jobmanager, following steps are required:
<ol>
<li>When installing the globus toolkit (when running globus-build
   command), you need to set your executable search path and other
   environmental conditions properly.  This can be done by executing
   the following command.
<pre>
   % source ${SGE_ROOT}/default/common/settings.csh
     or
   # . ${SGE_ROOT}/default/common/settings.sh
</pre>

<li>For the globus build, no additional options are required for
   enabling SGE in the globus environment.

<li>Before deploying the globus, you should edit the
   ${GLOBUS_INSTALL_PATH}/etc/globus-services.conf.  See the Globus
   Toolkit System Administration Guide, page 30.  For example, for SGE
   jobmanager on the host xxx.aist.go.jp, you would add the line:
<pre>
xxx.aist.go.jp jobmanager-grd stderr_log,local_cred - ${libexecdir}/globus-jobmanager \
globus-jobmanager -conf ${sysconfdir}/globus-jobmanager.conf -type grd -rdn jobmanager-grd \
-machine-type cluster
</pre>

<li>Deploy the globus by running globus-local-deploy command and start
   the gatekeeper.  These are regular procedures for running globus
   gatekeeper.

<li>In order to use SGE jobmanager, you have to specify the jobmanager
   as a gatekeeper contact.  Following is a sample command line
   sequence for contacting to SGE jobmanager:
<pre>
% globus-job-submit "xxx.aist.go.jp/jobmanager-grd" a.out
</pre>
</ol>

<b><font color="#0000ff">Enhancement of the job script for enabling jobarray submission:</font></b><br>
<b><font color="#0000ff">For Globus 1:</font></b><br>
The original version of the job script (globus-script-grd-submit.in)
does not support the jobarray submission.  In order to enable jobarray 
submission, we've slightly modified the job script for SGE jobmanager
in the globus package at:
<pre>
${GLOBUS_SRC_DIR}/ResourceManagement/gram/libraries/job_manager/globus-script-grd-submit.in
</pre>
Modified version of the job script is <A href="globus-script-grd-submit.in">here</A>.
In order to enable jobarray submission, you should replace this script
and re-install the globus toolkit.<br>

<b><font color="#0000ff">For Globus 2:</font></b><br>
In Globus 2, these jobscripts are included in Essential Grid
Services I bundle.
<pre>
globus_services_1_bundle.tar.gz
  -> globus_gram_job_manager-2.1.tar.gz
      -> scripts/globus-script-*-{submit,poll,rm}.
</pre>
<a href="globus-script-grd-submit">Here</a> is a job script for
globus2 package.  In order to enable jobarray submission, replace this
with the original version and re-install the globus toolkit.<br>

<b><font color="#0000ff">Detailed overview of this enhancement</font></b><br>
<pre>
----------------------------------------------------------------------
Globus supports the following four jobtypes.

jobtype 0 = mpi
jobtype 1 = single
jobtype 2 = multiple
jobtype 3 = condor

These jobtypes are assinged to grami_job_type variable by GRAM.
In order to enable jobarray submissions, I took the following
approach:

If jobtype ($grami_job_type) is single and count ($grami_count) is 1,
grd_jobtype is set to "single".
If jobtype ($grami_job_type) is single and count ($grami_count) is not
1, grd_jobtype is set to "jobarray".

Therefore, there could be the following five jobtypes:

# 5 jobtypes exist               GRD result
# -----------------              ------------------
# jobtype 0 = mpi         -----> run mpirun
# jobtype 1 = single      -----> submit one copy of grd script (if grami_count = 1)
# jobtype 1 = jobarray    -----> submit jobarray (if grami_count != 1)
# jobtype 2 = multiple    -----> submit count copy(s) of grd script
# jobtype 3 = condor      -----> ERROR

In order to submit jobarray through SGE gram, user have to specify
the resource like this:

% globusrun -r "host.xxx.yyy:/jobmanager-grd" \
'&(jobtype=single)(count=10)(executable=a.out)'

When the SGE gram is invoked on host.xxx.yyy, the script file for SGE
qsub command includes the following line:

#$ -t 10

When the user wish to specify the base index and stride for jobarray
submission, ex.

qsub -t 1001:1019:2

the user can specify them as environment variables like this:

% globusrun -r "host.xxx.yyy:/jobmanager-grd" \
'&(jobtype=single)(count=10)\
  (environments=(GRD_JOBARRAY_BASE 1001)(GRD_JOBARRAY_STRIDE 2))\
  (executable=a.out)'

In this case, the script file includes the following line:

#$ -t 1001:1019:2

1019 is automatically calculated by jobarray_base, count, and
jobarray_stride.

GRD_JOBARRAY_STRIDE is an optional and it's default value is 1.
----------------------------------------------------------------------
</pre>
    <hr>
    <address><a href="mailto:yoshio.tanaka@aist.go.jp">Yoshio Tanaka</a></address>
<!-- Created: Mon Feb  5 10:00:00 JST 2002 -->
<!-- hhmts start -->
Last modified: Mon Feb  5 10:00:00 JST 2002
<!-- hhmts end -->
  </body>
</html>
